Empirical Risk Minimization: Abstract Risk Bounds and Rademacher Averages

1. An abstract framework for ERM To study ERM in a general framework, we will adopt a simplified notation often used in the literature. We have a space Z and a class F of functions f : Z→ [0, 1]. Let P(Z) denote the space of all probability distributions on Z. For each sample size n, the training data are in the form of an n-tuple Zn = (Z1, . . . , Zn) of Z-valued random variables drawn according to some unknown P ∈ P. For each P , we can compute the expected risk of any f ∈ F by